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Abstract 

We show that the celebrated 1956 Lax-Richtmyer linear theorem in 
Numerical Analysis - often called the Fundamental Theorem of Nu- 
merical Analysis - is in fact wrong. Here "wrong" does not mean 
that its statement is false mathematically, but that it has a limited 
practical relevance as it misrepresents what actually goes on in the 
numerical analysis of partial differential equations. Namely, the as- 
sumptions used in that theorem are excessive to the extent of being 
unrealistic from practical point of view. The two facts which the men- 
tioned theorem gets wrong from practical point of view are : 

- the relationship between the convergence and stability of numerical 
methods for linear PDEs, 

- the effect of the propagation of round-off errors in such numerical 
methods. 

The mentioned theorem leads to a result for PDEs which is unrealis- 
tically better than the well known best possible similar result in the 
numerical analysis of ODEs. Strangely enough, this fact seems not to 
be known well enough in the literature. Once one becomes aware of 
the above, new avenues of both practical and theoretical interest can 
open up in the numerical analysis of PDEs. 
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1. TowEirds a correct relationship between stability and 
convergence 

It has been shown that in practically relevant situations the converse 
implication "convergent stable" in the Lax-Richtmyer theorem 
may fail to hold, see Rosinger [1-8], Rosinger & van Niekerk [1,2], 
Oberguggcnberger & Wang. Thus there need not always be an equiv- 
alence between the convergence and stability of a numerical scheme. It 
may therefore happen that convergence is a weaker property than sta- 
bility, which means that we may have convergent numerical schemes 
which nevertheless fail to be stable as well. 

In this way, what has become a kind of "UNIVERSALLY RECITED 
MANTRA" in the numerical analysis of linear partial differential equa- 
tions, namely that 

( ? ) "stability and convergence are equivalent" 

for linear numerical methods approximating such equations, does in 
fact lack a valid enough practical reality, and can be replaced with the 
far more convenient fact, according to which 

( ! ) "convergence need not always imply stability". 

Needless to say, the practical interest in such a possibility is signifi- 
cant, as it can enlarge the class of convergent numerical schemes be- 
yond those which are stable. Examples in this regard are mentioned 
in Rosinger [1-8], Rosinger & van Niekerk [1,2], Oberguggcnberger & 
Wang. 

By the way, the well known necessary condition for stability, given by 
von Neumann prior to the Lax-Richtmyer theorem, and which does 
not require a Banach space setup, can be seen as a further indication 
of the rather involved relationship between stability and convergence. 

A yet more important point about the above mantra is the follow- 
ing. Even if it were true in the linear case - which in fact is the only 
case addressed by the Lax-Richtmyer theorem - it would still lack rel- 
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evance in most of the cases when exact solutions of nonlinear PDEs 
are approximated by respective non/mear numerical methods. Indeed, 
as is well known, Kreiss, Stetter, a local linearized stability analysis 
of nonlinear PDEs and of their nonlinear numerical methods need not 
in general lead either to necessary, or sufficient convergence conditions. 

2. Questions about the implication "convergent stable" 

Briefly, the Lax-Richtmyer theorem, see below, states the equivalence 
between the convergence and stability of a linear numerical scheme 
which is consistent with a well posed hnear PDE, see Lax & Richt- 
myer, Richtmyer, Richtmyer & Morton, as well as a fully detailed 
analysis and presentation in Rosinger [2, pp. 1-14]. 

The important fact to note is the following. 

The proof of the imphcation "stable ^> convergent" is trivial, and 
certainly, it does in no way require the completeness of space in which 
it happens. 

Therefore, the crux of the Lax-Richtmyer theorem is solely in the proof 
of the converse implication, namely, "convergent =^ stable". 

That converse implication " convergent =^ stable" , however, is proved 
based on the celebrated Principle of Uniform Boundedness of Linear 
Operators in Banach spaces. And as is well known, see Appendix, that 
property of uniform boundedness does not necessarily hold in normed 
spaces which are not complete, thus fail to be Banach spaces. 

It is precisely here, with the assumptions which are made in order to 
secure a Banach space framework, that the Lax-Richtmyer theorem 
goes twice wrong from practical point of view. Namely, it goes wrong 
both with respect to the relationship between stability and conver- 
gence, as well as regarding the treatment of the essentially nonlinear 
phenomenon of the propagation of round-off errors. 

Furthermore, the proof of the implication " convergent =^ stable" is 
essentially linear, as it makes use of the mentioned linear principle, as 
well as of a hnear concept of stability. This makes the extension of 
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that implication to the fully nonlinear case extremely difficult. 

3. Is completeness an appropriate requirement ? 

The numerical analysis of a given PDE does typically assume the a 
priori knowledge of the existence of certain exact solutions of that 
equation. After all, in case an exact sohition does not exist, it is of 
course nonsensical to try to approximate it numerically. Thus, if and 
when the existence of an exact solution is known a priori, then the aim 
of numerical analysis is to construct numerical solutions approximat- 
ing one or another of such exact solutions. In this way, we are given 
an exact solution U and construct a sequence, say, llAt, with At > 0, 
of numerical solutions. Thus the problem is whether or not we have 
the hoped for convergence property 

( * ) limAt^o UAt = U 

where the limit holds in some appropriate sense. 

Suppose now, as usual, that both the exact solution U and the nu- 
merical solutions UAt belong to a certain normed space (X, 1 1 1 1 ) . 

A crucial observation here is the following one. And it is missed by 
the Lax-Richtmyer theorem. 

Clearly, in order to establish whether the above convergence property 
( * ) does, or for that matter, does not hold, one does not at all need to 
assume that the respective normed space {X, 1 1 1 1 ) is complete. Indeed, 
we have started by assuming that the exact solution U exists, thus the 
hoped for limit value in (*) exists. Furthermore, the terms Uai of the 
sequence in (*) also exist, being the constructed numerical solutions. 
Finally, the normed space X is supposed to be chosen in such a way 
that both the exact solution U and its numerical approximations UAt 
do belong to it. And then the only problem is whether the constructed 
numerical solution UAt does indeed happen to converge to the exact 
solution U. 

Furthermore, often, when for instance the exact solution [/ is a classi- 
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cal solution of the PDE considered, one can choose the normed space 
{X, II II ) as constituted from sufficiently smooth functions, since the 
numerical solutions UAt are typically defined at discrete points, thus 
they can be extrapolated to functions of required smoothness. 

In this way, there does not appear to be any practical reason whatso- 
ever why the normed space {X, \ \ || ) in which the convergence property 
(*) is to be established should be complete. 

The alleged reason why nevertheless the completeness of the normed 
space {X, II II ) is requested appears to be the claim that it is needed 
in order to handle the effect of round-off errors as well. Indeed, as 
it stands, the Lax-Richtmyer theorem is only supposed to deal with 
the effects of the propagation of truncation errors, since it does not 
anywhere mention directly round-off errors. 

However, as seen in Rosinger [5], see also Rosinger [2-4,7], this claim 
that the completeness of X will give the opportunity to deal as well 
with the effect of the propagation of round-off errors is simply unreal- 
istic from the point of view of the way round-off errors actually prop- 
agate in the computations involved. In particular, this claim leads to 
the paradox that one obtains a result regarding the effect of round-off 
errors in the numerical solution of PDEs which is strictly better than 
the well known best possible corresponding result in the case of the 
numerical solution of ODEs. 

Obviously, in the many usual cases when one approximates classical 
solutions U, one can choose the normed space {X, \ \ \\ ) made up of 
sufficiently smooth functions. But then, the completeness requirement 
in the Lax-Richtmyer theorem obliges one to consider its completion 
(X*, II II ). And typically, X"^ will be a much larger space, containing 
a considerable amount of non-smooth functions. 

Two important points should be noted here. 

First, within this larger and completed space X*, the original con- 
vergence problem ( * ) will remain precisely the same. Indeed, the 
constructed sequence of numerical solutions U^t & ^ converges to the 
existing exact solution U e X in the space X, if and only if it con- 
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verges to U in the space X*. 

On the other hand, the stability property of the respective numerical 
method may turn out to lead to a more stringent condition in the 
larger space X*, than in the original smaller space X. 

This is indeed of one the issues related to the assumption of com- 
pleteness, an assumption which is essential in the particular method of 
proof of the implication "convergent =^ stable" in the Lax-Richtmyer 
theorem. 

Otherwise, one simply notes that, in general, the completeness con- 
dition does not necessarily belong to the problem of establishing the 
convergence property (*). 

4. Compactness or Boundedness ? 

The above convergence relation ( * ), whenever it holds, clearly implies 
that the subset 

{Um I At > } U { t/ } C X 

is compact in. X, regardless of X being complete or not. And let us re- 
call that all the elements of this subset arc supposed to exist. Indeed, 
the exact solution U of the PDE under consideration exists, otherwise 
the problem of its numerical approximation would be vacuous. Fur- 
ther, the approximating numerical solutions C/a* are effectively con- 
structed by the numerical method employed. 

On the other hand, the condition of stability of the numerical meth- 
ods used in the Lax-Richtmyer theorem, see (5.18) below, is given in 
terms of boundedness, and as is well known, boundedness does not 
imply compactness in infinite dimensional normed spaces. 

This discrepancy between the association of convergence with com- 
pactness, and on the other hand, of stability with boundedness was 
first pointed out and dealt with in Rosinger [1], where with an appro- 
priate compactness based definition of stability, a general nonlinear 
equivalence result was given between convergence and stability. 
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It should be mentioned here that the above arguments related to sta- 
bility, convergence, completeness, compactness and boundedness were, 
back in the early summer of 1979, personally communicated by the 
author to P D Lax, at a conference at the Tel Aviv University, in Israel. 

5. Some details of the Lcix-Richtmyer theorem 

Let us now, for convenience, recall the Lax-Richtmyer theorem as 
given in its original formulation, see Lax & Richtmyer, Richtmyer, 
Richtmyer & Morton, Rosinger [2, pp. 1-14]. We consider a linear 
evolution type PDE 

(5.1) d/dtU{t) = A{U{t)), te[0,T] 
with the initial value 

(5.2) U{0) = u 

where A : D ^ X — > X is a linear operator defined on the subspace 
D of the Banach space X, u E D, while U : [0, T] — ^ D is the sought 
after solution. Since we deal with an evolution PDE, the operator A 
is in fact a linear partial differential operator in some space variable 

X e M". 

Further, one can assume that, when given, linear homogenous bound- 
ary conditions have already been incorporated in the definition of D. 
Typically, one can also assume that D is dense in X and we have 
satisfied the following exact solution property 

V ueD : 

3 U:[0,T] — > X : 

(5.3) 

*) limAt^o \\{U{t + At)-U{t))/At-A{U{t))\\ = 0, te[0,T] 
**) C/(0) = u 

Given now time, respectively space increments At e (0, oo) and Ax e 
(0, oo)", we construct a finite difference method 
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(5.4) CAt, Ax-.X ^ X 



which we assume to be a continuous hnear mapping. 

The numerical analysis problem we face in the above terms is to char- 
acterize the relations 

(5.5) Ax = a{At) 

where the mapping a : (0, oo) — > (0, oo)" is such that hmAt^o Oi(At) — 
e R", and the convergence property holds 

(5.6) limAt-^o, n->oo, nAt^t \ \ U{t) - C2^^^ u\\ = 

uniformly for t G [0,T], for every u & D, where U corresponds to u 
according to (5.3). 

As is well known, in general, this is not a trivial problem. In Courant 
et.al., it was shown for the first time that one cannot in general expect 
instead of (5.6) the stronger convergence property 

(5.7) limAt-^o, n-*oo, nAt-*t, Ax^O \ \ U{t) — C^^^ Ax 1 1 — ^ 

to hold uniformly for t € [0, T]. 

Property (5.7), in which At and Ax can simultaneously and indepen- 
dently tend to 0, is called unconditional stability. On the other hand, 
property (5.6), in which the relation (5.5) ties Ax to At when they 
both tend to 0, is called conditional stability. 

Obviously, in the case of conditional stability, one is interested in nu- 
merical methods (5.4) in which a tend to as fast as possible, when 

At tends to 0. Indeed, in such a situation one can obtain a good space 
accuracy without increasing too much the computation time. 

As a simple and immediate illustration, let us consider the initial value 
problem for the heat equation 
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Ut = U^x, t e [0,00), xeR 



U{0,x) = u{x), X eR 

In this case we can take {X, \ \ \\ ) — jC°°{R) and A — d'^/dx'^, while 

D = { M e £°°(M) nC2(M) I AueC°°{R)} 

and as is well known, John, the exact solution property (5.3) holds. 
A simple numerical method for this heat equation is given by 

(CAt,Axu)(x) = u{x) + {u{x + Ax)-2u{x)+u{x-Ax))At/Ax'^ 

for M e X, X E R, At, Ax > 0. Also as is well known, Richtmyer, 
this explicit numerical method will not have the convergence property 
given by the unconditional stability (5.7), while the weaker conver- 
gence property (5.6), called conditional stability, will hold, if and only 



2At < Ax^ 
which in terms of (5.5) can be written as 

a{At) > ^/{2At) 

This obviously means that a — > rather slowly, when At — > 0, which 
is inconvenient, since a small Ax will impose the use of a quadratically 
smaller At, leading thus to an increased number of time iterations. 

Returning to the general case, in view of the exact solution property 
(5.3), we can define the family of linear mappings 



if 



(5.8) Eo{t) : D 



X, te[o,T] 



by 



(5.8.1) Eo{t){u) 



U{t), te[0,T] 
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The initial value problem (5.1), (5.2) is called properly posed, if and 
only if the family of linear mappings (5.8) is uniformly bounded, that 
is, for a certain > 0, we have 

(5.9) \\Eo(t)\\<K, te[0,T] 

As is well known, since D is a dense subspace in X, one can extend 
by continuity the family of linear mappings (5.8) to a unique family 
of linear mappings 

(5.10) E{t) :X — > X, te[0,T] 
with the same uniform bound, namely 

(5.11) II E{t) \\<K, te[0,T] 

In addition, we shall have the semigroup property 

E{t)E{s) = E{t + s), t,s e[0,T], t + s <T 

(5.12) E{0) = idx 

limAt^o II E{u) — M II = 0, u E X 

We can note that the semigroup property (5.12) leads to a further 
extension, this time of the linear mappings (5.10), namely 

(5.13) E{t):X — > X, ie[0,oo) 
where 

(5.13.1) E{t) ^ E{t-[t/T]T)E{T)WT]^ te[T,oo) 

with [t/T] denoting the largest integer which is smaller than, or equal 
to t/T. In this case, instead of the corresponding above relations, we 
shall have 
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E{t)E{s) = E{t + s), t,se [0,00) 

(5.14) 

\\ E{t) \\ < K^+^'/^\ te[0,oo) 

Returning now to the numerical method (5.4), (5.5), we shall consider 
as our finite difference scheme the family of continuous linear mappings 

(5.15) Ca* = Ca*, a{At) '■ X )• X 

Here it is important to note that, typically, this family of continuous 
linear mappings Ca*, with At > 0, is not uniformly bounded for small 
At. 

Now in view of (5.5), (5.6), the finite difference scheme (5.15) is called 
convergent to the semigroup (5.10) - (5.12) on the time interval [0, T], 
where T > is given, if and only if 

V ueX, e>0 : 
3 5>0 : 

(5.16) 

V t e [0,T], At > 0, n e N, uAt < T : 

At, \t-nAt\<S =^ II E{t) - C\^u || < e 

Further, the finite difference scheme (5.15) is called consistent with 
the initial value problem (5.1), (5.2) on the same time interval [0,7], 
if and only if 

V ueX, e>0 : 
3 e>0 : 

(5.17) 

V t e [0,T], At > : 

At<e =^ \\CAtE{t)-E{t + At)u\\<e 
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Finally, the finite difference scheme (5.15) is called stable on the time 
interval [0, T], if an only if 



3 L>0 : 



(5.18) V At>0, neN, nAt<T 



Clt\\<L 



With the above, we have the so called Fundamental Theorem of Linear 
Numerical Analysis 

Theorem ( Lcix-Richtmyer, 1956) 

Given a properly posed semigroup (5.10) - (5.12) and a finite differ- 
ence scheme (5.15) which is consistent with it, then the finite difference 
scheme is convergent to the semigroup, if and only if it is stable. 

Remcirk 

The practical interest in the above type of result is in the follow- 
ing. The consistency of a finite difference scheme with a semigroup 
generated by an initial value problem (5.1), (5.2) is typically easy to 
establish with the use of a finite Taylor series argument, in case we deal 
with smooth enough, or classical solutions. Also, what is practically 
particularly important, the consistency property can be established 
without the effective knowledge of any specific exact solution of the 
initial value problem, and only based on the knowledge of the regu- 
larity of such solutions, that is, the existence of smooth enough, or 
classical exact solutions. 

The convergence property of such a finite difference scheme is, of 
course, the main and nontrivial issue, and just like the consistency 
property, it is a relational property, since it involves the semigroup, 
or the initial value problems as well. Furthermore, here the fact that, 
typically, the exact solution is only known to exist, but it is not known 
effectively - this being the very reason for using numerical analysis - 
makes it so much more difficult to establish convergence. 
On the other hand, the stability property of a finite difference scheme 
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is no longer a relational property, but an intrinsic property which is 
solely of the finite difference scheme itself, therefore, at least in prin- 
ciple, it can be established alone on the information contained in that 
finite difference scheme. 

In this way, in the study of the convergence of finite difference schemes 
there is clearly a major interest in establishing a certain connection 

between the relational property of convergence which is the sought af- 
ter aim, and on the other hand, the intrinsic property of stability. 

The above Lax-Richtmyer theorem does estabhsh such a connection, 
in fact, an equivalence, between convergence and stability. Unfortu- 
nately however, it assumes the completeness of the normed space in 
which all of this happens, in order to be able to prove the implication 
"convergent =^ stable". 



Appendix 



We present a simple counterexample to the celebrated Principle of Uni- 
form Boundedness of Linear Operators in a Banach Space, based on 
the fact that the respective normed space fails to be complete, that 
is, Banach. This shows that in this principle, the completeness of the 
normed space involved is indeed essential. 



) defined as follows 



3 m G N : 
V n E N, n> m : 







We take the normed space (X, 

(A.l) X = {x = {xo,Xi,X2, 

with the norm given by 

(A.2) = sup { | e N } 

for X — {xq, Xi, X2, . . . ) & X. 

Now, for every /c G N, we define the linear operator : X 
(A.3) Tfe(xo,xi,X2, ■ ■ ■ ) = (yo,yi,y2, ■ ■ ■ ) 



A by 
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where for x — (xq, a^^i, 2^2, ■ ■ ■) & X,we have 
kxk a n = k 

(A.4) 7/„ = 

\i n^k 
It follows easily that 

(A.5) II Tfc II = A;, A; e N 

therefore, the family of linear operators (T^t | A; G N) is not uniformly 
bounded. 

On the other hand, given any fixed x = {xo,Xi,X2, . . . ) E X, there 
exists m G N, such that a;„ = 0, for n & N,n > m. Hence Tk{x) — 0, 
for A; e N, A; > m. Consequently 

(A.6) sup {II Tk{x) II I keN} < 00 

In this way, the family of linear operators (T^ | k G N) is bounded at 
each point x E X, and yet it is not uniformly bounded on X. 

The reason for that is obviously in the fact that the normed space 
(X, II II ) is not complete. Indeed, (X, 1 1 1 1 ) is a strict and dense 
subspace of 
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